A new algorithm to build feed forward neural networks

The paper presents a new algorithm to build a feedforward neural network with a single hidden layer. The algorithm starts with 1 hidden unit and the new hidden units are added to the network only if they improve the classification accuracy of the network on the cross-validation samples. The initialization of the weights and bias is done using Nguyen-Widrow method and the network is trained without freezing the weights. The results show that the training accuracies obtained using this algorithm are better than that obtained using N2C2S algorithm. The crossvalidation accuracies and the test prediction accuracies obtained by using both the algorithms are not statistically significantly different. Due to this and also since it is easy to understand and implement than N2C2S algorithm, the proposed algorithm should be preferred than the N2C2S algorithm. Along with this 3 different methods of obtaining weights for neural networks are also compared. The classification results obtained using this algorithm are compared to the prediction accuracies obtained using logistic regression, C5.0 and M5′ classification techniques on 5 freely available data sets. The classification results show that NN is better than logistic regression over 2 data sets, equivalent in performance over 2 data sets and has low performance than logistic regression in case of 1 data set. NN is better than C5.0 over 1 data set and equivalent in performance on the remaining data sets. It is observed that M5 is a better classification technique than other techniques over 1 dataset. M5′ is better than logistic regression over 2 data sets and equivalent in performance for the remaining datasets. C5.0 is better than logistic regression over 1 data set and equivalent in performance on the remaining data sets. Keywords-classification, logistic regerssion, feedforward neural network, backpropagation.